What is a P-value?

Results favoring one treatment over another in a randomized clinical trial can be explained only if the favored treatment really is superior or the apparent advantage enjoyed by the treatment is due solely to the working of chance. Since chance produces very small advantages often but large differences rarely, the larger the effect seen in the trial the less plausible chance assignment alone can be as an explanation. If the chance explanation can be ruled out, then the differences seen in the study must be due to the effectiveness of the treatment being studied. The p-value measures consistency between the results actually obtained in the trial and the “pure chance” explanation for those results. A p-value of 0.002 favoring group A arises very infrequently when the only differences between groups A and C are due to chance. More precisely, chance alone would produce such a result only twice in every thousand studies. Consequently, we conclude that the advantage of A over B is (quite probably) real rather than spurious. What does it mean when the results of a randomized clinical trial comparing two treatments reports that, “Treatment A was found to be superior to Treatment C (p = 0.002)”? How much, and how little, should non-statisticians make of this? The interpretation of the p-value depends in large measure on the design of the study whose results are being reported. When the study is a randomized clinical trial, this interpretation is straightforward. Conducting the ideal study The ideal study to compare two treatments — an active drug and a placebo, for example — would be to test each treatment in a laboratory experiment that would be identical aside from whether Compound A or Compound C was under ∗Professor, Departments of Statistics, Health Studies, and Anesthesia & Critical Care,The University of Chicago, 5841 South Maryland Avenue (MC 2007), Chicago, IL 60637. URL:. c © 1998 Ronald A. Thisted